Recently, a machine learning using a neural network having a multi-layer structure has attracted attention. Such a machine learning using a neural network having a multi-layer structure is also called “deep learning”. In the deep learning, a multi-hierarchization of the neural network has proceeded, and effectiveness thereof is confirmed in many fields. For example, the deep learning exerts a high recognition accuracy, which is comparable to that of human beings, in recognizing an image and a sound.
Patent document 1: Japanese Laid-open Patent Publication No. 2008-310524
The deep learning performs a supervised learning to cause the neural network to automatically learn features. However, a used memory amount of the deep learning is large because of the multi-hierarchization of the neural network, and thus the used memory amount is more increased in learning. For example, an error backward propagation method, which is commonly used in the supervised learning, causes the neural network to propagate forward data for learning to perform recognition, and compares a recognized result with a correct answer to obtain errors. Moreover, the error backward propagation method causes the neural network to propagate the error from the correct answer of the recognized result in the inverse direction of that at the recognition so as to change parameters of respective hierarchies of the neural network. Thus, the used memory amount increases in learning. For example, because gradients of the error are saved in the learning, a data amount increases to more than twice of that in a case where only the recognition is performed, and the used memory amount increases in some cases to more than twice.